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Abstract 

Despite the ever-increasing number of online professional development 
(OPD) programs, relatively few studies have been conducted to examine the 
efficacy of such programs for teachers and students. This manuscript presents 
findings from an impact study of OPD courses infractions, algebraic thinking 
and measurement on 79 fifth grade teachers’ pedagogical content knowledge 
and pedagogical practices as well as their students’ mathematics achievement. 
The OPD courses were offered one course per semester for three semesters, 
and each course comprised 1 week of orientation and 6 weeks of course con¬ 
tent. Overall, teachers participated in more than approximately 70 hours of 
OPD. The research findings showed that teachers who had been randomly 
assigned to the experimental group had significantly greater gains in scores 
for pedagogical content knowledge and pedagogical practices than teachers in 
the control group. Nevertheless, the positive changes in teacher outcomes did 
not translate to any meaningful differences in students’ mathematics achieve¬ 
ment. (Keywords: teacher quality, online professional development, elemen¬ 
tary education, mathematics achievement) 


The single thing that determines how well a child does in math is ... an 
outstanding math teacher. 

—President Barack Obama (Burt-Murray, Robertson, & Gordy, 2010) 


I t has been well established in the research literature that highly quali¬ 
fied and highly effective teachers are key to students’ academic success 
(Darling-Hammond & Berry, 2006; Geringer, 2003; Lasley, Siedentop, & 
Yinger, 2006). In a review of John Hattie’s seminal work, Visible Learning: 

A Synthesis of Over 800 Meta-Analyses Relating to Achievement, Terhart 
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(2011) critically analyzed Hatties (2009) claim that the “teacher” factor was 
the most significant school predictor of students’ academic achievement, 
particularly in the area of mathematics. Despite criticism of some of Hattie’s 
methods, Terhart (2011) determined that teachers do matter, as only certain 
teachers with certain practices are highly effective. Indeed teacher quality 
has been cited as the single most important school factor that explains stu¬ 
dent achievement (Hanushek, 2007; Haskins & Loeb, 2007; Haycock, 2003; 
Gordon, Kane, & Staiger, 2006). Still, more than two decades after the publi¬ 
cation A Nation at Risk (U.S. Department of Education, 1983), challenges re¬ 
garding teacher quality continue to be both poignant and relevant (Peterson, 
2003; U.S. Department of Education, 2008). Specifically, the National Com¬ 
mission on Excellence in Education highlighted severe shortages of teachers 
in the areas of mathematics and science in the 1983 publication A Nation 
at Risk. Moreover, the report indicated that half of the newly employed 
mathematics teachers were not actually qualified to teach the subject. Some 
argue that little has changed in the years following the report and education 
outcomes have shown little improvement since 1970 (Peterson, 2003). Fur¬ 
thermore, there is little evidence to indicate that the highly qualified teacher 
provisions of the No Child Left Behind (NCLB) Act have led to significant 
increases in teacher efficacy or to increases in teachers’ knowledge in subject 
areas such as mathematics (U.S. Department of Education, 2008). 

Professional development for teachers has been deemed the necessary ap¬ 
proach to improving teacher quality, meaning teachers’ pedagogical content 
knowledge and pedagogical practices. Essentially, professional development 
has been adopted as a policy solution to improving the number of highly 
qualified teachers as well as helping all students to achieve high academic 
standards (Colbert, Brown, Choi, & Thomas, 2008). Guided by the mandate 
that all students should be taught by highly qualified teachers, the NCLB 
Act stipulates that any local educational agency that receives Title I, Part A, 
funds must provide their teachers with high-quality professional develop¬ 
ment (Public Law 107-110-JAN.8, 2002). Support for this policy is based 
on the premise that teacher knowledge and classroom practices mediate the 
effects of professional development on student achievement, provided the 
professional development is conducted within the context of high standards, 
challenging curricula, systemwide accountability, and high-stakes assess¬ 
ments (Yoon, Duncan, Lee, Scarloss, & Shapley, 2007). 

The Wenglinsky (2000) report, How Teaching Matters: Bringing the 
Classroom Back into Discussions of Teacher Quality, documents the positive 
effects of teacher professional development. Specifically, Wenglinsky (2000) 
provided evidence that students whose teachers had received professional 
development in working with special populations outperformed their peers 
on mathematics assessments by more than a full grade level. Moreover, 
students whose teachers had received professional development in higher- 
order thinking skills outperformed their peers on mathematics assessments 
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by 40% of a grade level (Wenglinsky, 2000). Further evidence in the research 
literature indicates that teachers who receive substantial professional devel¬ 
opment (an average of 49 hours) can boost their students’ academic achieve¬ 
ment by approximately 21 percentile points (Yoon, Duncan, Lee, Scarloss, 

& Shapley, 2007). According to Yoon et al. (2007), professional development 
is most effective if it is characterized by coherence, active learning, suffi¬ 
cient duration, collective participation, a focus on content knowledge, and a 
reform rather than traditional approach. 

Despite these guidelines, the U.S. Department of Education (2007) 
publication “State and Local Implementation of the No Child Left Behind 
Act Volume II— Teacher Quality Under NCLB: Interim Report” indicates 
that few teachers receive content-focused professional development for a 
sufficient duration. Specifically, although almost all elementary teachers re¬ 
ported participating in content-focused professional development related to 
teaching mathematics, only 8% of those teachers had received more than 24 
hours of professional development on instructional strategies in mathemat¬ 
ics (U.S. Department of Education, 2007). The unavailability of professional 
development opportunities that directly address teachers’ individual needs 
and the rigid schedule associated with traditional professional development 
programs are some of the obstacles that many teachers face (Kleiman, 2004). 
Traditional professional development is often offered at times and locations 
that are not feasible for many teachers, and the reality of having to meet the 
demands of both work and family life require professional development that 
can be delivered anytime, anywhere (Vrasidas & Zembylas, 2004). 

Online professional development has been championed as the “anytime, 
anywhere” option that provides flexibility by allowing participants, irrespec¬ 
tive of location, to manage educational pursuits with work and personal 
responsibilities (Davis, 2009; Stanford-Bowers, 2008). Born out of a need for 
professional development that fits with the busy schedules of teachers, on¬ 
line professional development provides access as well as ongoing support to 
important resources that might not otherwise be affordable or even available 
locally (Dede, Breit, Ketelhut, McCloskey, & Whitehouse, 2005). The advent 
of online courses makes it feasible to connect teachers across schools, dis¬ 
tricts, and even states, thereby fostering professional learning communities 
and broadening the professional learning opportunities available to teachers 
(Russell, Carey, Kleiman, & Venable, 2009). Additionally, online professional 
development can be offered in various forms: Distance learning classrooms 
enable individuals to participate in a class via video conferencing with the 
goal of making the online experience as close as possible to an in-class expe¬ 
rience; an online course can be conducted completely through asynchronous 
interactions, negating the need for all the participants of a course to be avail¬ 
able at the same time and allowing participants to complete course require¬ 
ments according to their individual schedule; and self-paced online courses 
allow each participant to work through a series of resources and activities 

Volume 45 Number 1 | Journal of Research on Technology in Education | 3 
Copyright © 2012, ISTE (International Society for Technology in Education), 800.336.5191 
(U.S. & Canada) or 541.302.3777 (Int'l), iste@iste.org, iste.org. All rights reserved. 



Dash, Magidin de Kramer, O'Dwyer, Masters, & Russell 


at his or her own pace (Russell et al„ 2009). Ultimately, online professional 
development provides the potential means of meeting the ambitious goals of 
NCLB regarding teacher quality, whereby large numbers of teachers can be 
provided with high-quality training in mathematical content and pedagogy 
(Ginsburg, Gray, & Levin, 2004). 

The advantages associated with online professional development have 
been linked with a dramatic increase in such programs nationally (Carey, 
Kleiman, Russell, Venable, & Louie, 2008). However, this increase in the 
number of online professional development programs has been coupled with 
increased concerns regarding the efficacy of online professional develop¬ 
ment as a strategy to improve educational outcomes (Carey et al„ 2008). In a 
review of 40 online professional development sites for teachers of mathemat¬ 
ics, Ginsburg, Gray, and Levin (2004) found no independent evidence of 
program effectiveness. Moreover, the ultimate worth of any teacher profes¬ 
sional development initiative is often gauged by whether there is evidence of 
an impact on teacher practices and student learning (Lawless & Pellegrino, 
2007). Previous research studies have documented evidence regarding vari¬ 
ous designs and methods of delivering online professional development. 
However, there is currently a void in the research literature regarding the 
effects of online professional development on teacher quality and student 
achievement (Ginsburg, Gray, & Levin, 2004; Lawless & Pellegrino, 2007; 
Whitehouse, Breit, McCloskey, Ketelhut, & Dede, 2006). 

Research Purpose 

The purpose of this study was to investigate the effects of online professional 
development in mathematics on fifth grade teachers’ pedagogical content 
knowledge and pedagogical practices as well as their students’ mathematics 
achievement. 

We conducted this study as part of a larger research study associated with 
a specific online professional development program. The online professional 
development program is a federally funded 10-state collaborative that was 
founded on the premise that many teachers do not have ready access to 
high-quality professional development. Thus, the program seeks to create 
an effective and sustainable model of online professional development for 
teachers in each of the 10 participating states by removing schedule and 
location barriers and offering teachers the opportunity to pursue profes¬ 
sional development courses online. The ultimate program goal is to improve 
teachers’ pedagogical content knowledge and instructional practices as well 
as students’ academic achievement. As an important part of the initiative, we 
conducted four concurrent experimental studies in fourth grade English lan¬ 
guage arts (ELA), fifth grade mathematics, seventh grade ELA, and eighth 
grade mathematics (O’Dwyer, Masters, Dash, Magidin De Kramer, Humez, 
& Russell, 2010). The focus of this paper is the fifth grade mathematics ran¬ 
domized controlled trial. 
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Structure of the Fifth Grade Mathematics Online 
Professional Development Program 

The online professional development courses described in this study were 
three elementary mathematics courses developed by the Education Devel¬ 
opment Center Inc. (EDC) for use with teachers of grades 3-5. The fifth 
grade mathematics online professional development program comprised 
three courses: Using Models to Understand Fractions, Algebraic Thinking in 
Elementary School, and The Complexities of Measurement. One course was 
offered each semester for three semesters, and each course comprised 1 week 
of orientation and 6 weeks of course content. We estimate that each course 
required 4-6 hours of work per week from each teacher. Therefore, teach¬ 
ers participated in more than approximately 70 hours of online professional 
development related to teaching specific topics within mathematics. 

The online professional development courses were asynchronous, in 
that, unlike a virtual classroom, course participants were not required to 
be available at the same time. This asynchronous learning allowed teachers 
to complete course requirements according to their individual schedules. 

The courses were also facilitated by individuals who were first trained in 
online course facilitation by EDC/EdTech Leaders Online (ETLO). ETLO 
is a capacity-building program administered by EDC that enables state 
departments of education, school districts, teacher training institutions, and 
other educational organizations to provide effective online learning pro¬ 
grams for teachers. The EDC/ETLO-trained online course facilitators were 
responsible for monitoring teachers’ completion of course activities and 
issuing certificates of completion to teachers who completed all of the course 
requirements. Moreover, the courses used a learning community model that 
included readings, web-based resources, interactive on and offline activities, 
video, and peer-to-peer discussions. Each course culminated with a project 
that teachers were required to implement in their classroom. The theoreti¬ 
cal underpinning of each course was that teachers would be provided with 
tools, strategies, and opportunities to explore engaging mathematical activi¬ 
ties that could be integrated into their classroom instruction. Furthermore, 
as teachers improved their knowledge and skills in teaching mathematics, 
they would be better able to help students improve their own understanding 
of mathematical concepts. 


Interventions 


The Fractions Intervention 

The fractions course began with an exploration of different ways to think 
about fractions problems and a review of how fractions are taught in el¬ 
ementary school (see Figure 1 on p. 6). In the second and third sessions, par¬ 
ticipants departed from using traditional algorithms and investigated how 
linear measurement models can be used to promote a deeper understanding 
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Orientation 

This orientation session should be completed prior to the start of our course. It includes an introduction to the course website and on 
opportunity to Introduce yourself to fellow participants as you experiment with the online discussion board 




□ 

Is there only one ‘right* way to solve fractions problems? In the first session of this course, you win focus on how students In grades 3-5 
think about fractions. 



Not quite gne meter; v >inti linear mcn>ufcnicnl nigtlcb.LpJiitr.gd me Imiioiia 

How do we describe the length of an object when It Is longer than one benchmark number on a ruler, but shorter than another? This session 
explores the Importance of using a linear measurement model, such as a number line or ruler, to think about fractonal quantities. 


_ 


Figure 1. Screen shot of the Assignments page of the Fractions course. 


of fractional relationships. During sessions 4 and 5, participants examined 
area models, specifically pattern blocks. Throughout the course, participants 
were given opportunities to apply linear and area models to actual fractions 
problems, use virtual manipulatives, look at student work (to shed light on 
how young students think about fractions), and discuss their experiences 
and ideas with their colleagues. By the sixth and final session, participants 
completed a final project and a student interview. Examples of activities in 
the Fractions course are described below: 

Fractions problems such as the four listed below are often given to stu¬ 
dents in grades 3-5. In a moment you will be analyzing samples of student 
work for these problems, so take a few moments to solve each problem 
first. Think about the following questions as you work through them: 

1. Are you using algorithms (e.g., mathematical procedures for solving 
the problem) or mental math strategies (e.g., using estimation, round¬ 
ing, or identifying numerical relationships) to solve these problems? 

2. Can you solve these problems in more than one way? 

3. Reflect on the issues depicted in the reading by Carne Barnett. Do 
these problems extend students’ understanding of fractions, or do they 
illustrate their ability to calculate an answer using an algorithm? 

The Algebraic Thinking Intervention 

The Algebraic Thinking course began with an exploration of what algebraic 
reasoning means and exploring activities such as identifying patterns, ana¬ 
lyzing change, representing situations, and using mathematics models to un¬ 
derstand relationships as algebraic (see Figure 2). During the second session, 
participants examined types of problems that encourage algebraic thinking 
and how generalization can help solve some algebraic problems. The third 

6 I Journal of Research on Technology in Education | Volume 45 Number 1 
Copyright © 2012, ISTE (International Society for Technology in Education), 800.336.5191 
(U.S. & Canada) or 541.302.3777 (Int'l), iste@iste.org, iste.org. All rights reserved. 











Impact of Online Professional Development 



EdTech 

LEADERS* 

n.—.1.71 gj 

*. H — n 

tdr«J. londw. 





□ 


This session Includes a look at NCTM's Algebra Standard fo« Grades 3-5 and readings that explain why teaching algebraic concepts to 
elementary students Is important. You wlB also engage In some algebra problem solving and have an online discussion about what algebraic 
thinking can look like In elementary school 


This session will focus on one type of algebraic problem: using generalizations. Learning to generalize about a pattern Is an Important part of 
algebraic thinking, but generalization does not work to solve all algebra problems. In both the readings and the activities durlrg this session. 
' you wll see that there Is no one ’right* way to solve algebraic problems, and that each sokAion method contributes some new richness to 
the problem. 


Hie Mcaninu gLLuudlttY 

In this session, you will consider why young students have a difficult time solving algebraic problems. The readings will familiarize you with 
current research about young students' Ideas about equality, and you will solve some problems that require you to think flexibly about the 
role of the equals sign In a number sentence. _ 


Figure 2. Screen shot of the Assignments page of the Algebraic Thinking course. 


session introduced participants to students’ misconceptions about the = 
sign, and participants were taught instructional strategies to help students 
better understand what the symbol means. Sessions 4 and 5 were designed 
to show participants that algebraic thinking does not always involve find¬ 
ing a missing quantity and to teach participants how to promote algebraic 
thinking. By the end of the sixth session, participants explored additional 
strategies to incorporate algebra into their instruction and completed a final 
project. Examples of activities in the Algebraic Thinking course are de¬ 
scribed below: 

The activities that follow present you with some different types of equal¬ 
ity problems. There are a number of ways to promote algebraic reasoning 
about the symbol =, and it is worth noting that none of these problems 
contain the letter x! As you solve these, think about what assumptions you 
are making about the symbol =. When does the symbol = indicate that 
computation can take place, and when is it used as a balance between two 
sides of an equation? Later on in this session, you will have the chance to 
watch some videos of students solving missing value problems. You will 
also examine some student work to see some examples of the types of er¬ 
rors that students make. 

The Measurement Intervention 

The Measurement course began with an overview of the importance of study¬ 
ing measurement in upper elementary school and investigating some common 
student misconceptions on measurement (see Figure 3 on page 8). The second 
and third sessions introduced participants to two-dimensional measurement 
with linear measurement and area. Additionally, participants investigated the 
importance of iteration and unitization. Session 4 focused on three-dimen¬ 
sional measurement by looking at student understanding of capacity. During 
Session 5, instructors asked participants to consider measuring nonstandard 
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In thH M“.Mon you mill read about some ol the reasons students struggle with this distinction and, through the use o i video, you « 
some 4th grade students working to understand area as they partkipate in a classroom lesson 


In this session, you will explore capacity and think about how the teaching ot capacity tits in with an overall study ot measurement. 


5 ' I Mcttturina Ignmlimtcd Qbi«.la 

I In thiv session, you will reflect upon how measuring complicated objects can help demonstrate a need for measurement (and measurement 
| conventions) In real Me. 


Figure 3. Screen shot of the Assignments page of the Measurement course. 


objects and how to help students understand the concepts. During the final 
session, participants had the opportunity to reflect on what they had learned 
and to integrate it into their classrooms. Examples of activities in the Mea¬ 
surement course are described below: 

In the video below, teacher Barbara Smith reflects on the way that she 
teaches measurement in her class. She also discusses the ideas that her stu¬ 
dents had as they worked on the area and measurement activities earlier in 
this course. As you watch the video, think about the following questions: 

1. What measurement ideas does she stress in her instruction? 

2. What implications does that have for student learning? 

3. To what extent does her instruction include two of the central ideas of 
this course: using consistent measurement units and learning to iterate? 

Theory of Change 

The fifth grade mathematics online professional development program 
was grounded on the principles that professional development in fractions, 
algebraic thinking, and measurement would lead to improvements in teach¬ 
ers’ knowledge of how to teach these subject areas to fifth grade students. 
Furthermore, improvements in teachers’ knowledge would lead to improved 
instructional practices, which in turn would lead to improved mathematics 
achievement for students. Specifically, the theoretical underpinnings of each 
course is that teachers will be provided with tools, strategies, and opportuni¬ 
ties to explore engaging activities in mathematics that they could integrate 
into their classroom instruction. Furthermore, as teachers improve their 
knowledge and skills in teaching mathematics, they would be better able to 
help students improve their own understanding and achievement. Consistent 
with the models of effective professional development described in Yoon, 
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Duncan, Lee, Scarloss, and Shapley (2007), this study is based on the theory 
that: (a) online professional development will increase teachers’ pedagogical 
content knowledge, (b) increased pedagogical content knowledge will lead to 
improvements in teachers’ instructional practices, and (c) improved instruc¬ 
tional practices will result in improved academic achievement for students. 

The pedagogical content knowledge construct was first introduced by 
Shulman (1986), who argued that content knowledge and pedagogical 
knowledge are not separate and distinct domains because pedagogical con¬ 
tent knowledge goes beyond mere knowledge of subject matter. Specifically, 
pedagogical content knowledge pertains to knowledge of subject matter for 
teaching—that is, knowledge of ways to represent and formulate the subject 
matter that make it comprehensible to students. Good teachers know both 
content and how to teach that content to their students (Hill & Ball, 2009). 
Pedagogical content knowledge includes an understanding of what makes 
the learning of specific topics easy or difficult, the conceptions and miscon¬ 
ceptions that students might have, and the strategies most likely to be effec¬ 
tive in reorganizing the understanding of students (Shulman, 1986). Thames 
and Ball (2010) further posit that conventional content knowledge is insuf¬ 
ficient to effectively teach K-8 mathematics. The mathematics knowledge 
that teachers need for instruction is not the same mathematics taught and 
learned in college classes (Ball, Thames, & Phelps, 2008). Rather, teachers 
require specialized knowledge of how to teach mathematics (Thames & Ball, 
2010). Teachers require pedagogical content knowledge in mathematics. 

Methods 


Research Design 

This study employed a randomized controlled trial that we implemented 
in three rounds between January 2007 and June 2009. Fifth grade teach¬ 
ers who taught mathematics to a regular-education fifth grade class (for 
the duration of the research study) qualified to participate in the study. 
Teachers volunteered to participate in the research study as they typically 
would for a professional development workshop. Teachers were recruited 
by each of the eight original participating states as well as through 
listservs and other online communities. Overall, the sample of teachers 
represented 12 states. 

For each round of implementation, we randomly assigned fifth grade 
mathematics teachers to either a control or experimental group. Teachers 
in the experimental group completed a series of three online professional 
development mathematics courses, offered one course per semester. It is im¬ 
portant to note that though teachers assigned to the control group were not 
allowed to participate in the online professional development courses, they 
were not restricted from participating in any other professional development 
that might otherwise have been available to them. 
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We collected data from both students and teachers over three periods 
of data collection. Although teachers’ participation lasted for three school 
semesters, the student participants were drawn from only two school semes¬ 
ters, the fall and spring semesters of the teachers’ second year of participa¬ 
tion. The first round of the randomized trial started in the spring semester of 
2007, which meant that we could not measure student achievement over an 
entire school year for the teachers’ first class of students. Therefore, we col¬ 
lected student data from the teachers’ second class of students. Prior to the 
start of any professional development, all teachers were required to complete 
a background survey as well as a presurvey regarding their pedagogical con¬ 
tent knowledge and practices in mathematics. 

The second period of data collection occurred in the fall semester after 
the teachers in the experimental group had completed the first online 
professional development course. We administered no teacher instruments 
during the second data collection period, but teachers were required to ad¬ 
minister the student background survey and a student mathematics pretest 
to their new fifth grade class. 

The third data collection period occurred at the end of the academic year, 
after the teachers in the experimental group had completed the remaining 
two professional development courses. During this period, all teachers were 
required to complete a postsurvey that focused on their pedagogical content 
knowledge and pedagogical practices in mathematics. In addition, teachers 
administered the student posttest measures. 


Sample 

Teacher participants. We recruited a total of 235 teachers for this study. 
However, during the course of the study, 143 teachers resigned from partici¬ 
pation (61% of recruited sample). It should be noted that distance teaching 
and learning environments are particularly vulnerable to high attrition, 
with dropout rates typically ranging from 30% to 50% (O’Brien & Renner, 
2002; Stanford-Bowers, 2008). However, the attrition rate for this study was 
even higher than the norms indicated in the research literature because 
participants were required to commit to the study for three semesters. 
Unfortunately, the vast majority of participants did not provide reasons for 
their withdrawal from the study. However, some participants did indicate 
that they were no longer able to participate due to personal challenges such 
as illness in the family. Nevertheless, we conducted chi-square analyses to 
determine whether there were significant differences between teachers who 
dropped out of the study and those who remained. The results indicated that 
significantly more teachers who dropped out of the study (67%) had no pre¬ 
vious training with online professional development, compared with teach¬ 
ers who remained in the study (44%), y 2 (3) = 13.59, p = .004. The effect size 
associated with this difference as measured by Cramer’s V was .261, which, 
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according to Cohens (1988) criteria, is a medium effect. There were no other 
significant differences in demographic information between teachers who 
remained in the study and those who dropped out. 

The sample included 13 pairs of teachers who worked at the same 
school. In an effort to avoid any confounding school effects, we assigned 
teachers from the same school to the same research group (experimental 
or control) and randomly selected one teacher from each of the 13 pairs 
for inclusion in the data analysis. This procedure resulted in the removal 
of data from 13 teachers (6% of recruited sample) and negated the pos¬ 
sibility of teachers from the experimental and control groups working 
together. It should also be noted that after random assignment, with the 
exception of one pair of teachers, we assigned all of the duplicate teach¬ 
ers to the control group. Ultimately, the results of the study were based 
on a sample of 79 teachers, including 34 experimental group teachers 
and 45 control group teachers. 

The teacher sample was predominantly White (89%) and female (90%). 
More teachers reported that they were in the age groups 26-35 (33%) and 
46-55 (30%) than any other age category. The majority of the teachers had 
regular state certification (91%). Fifty-eight percent of the teacher partici¬ 
pants reported having a master’s degree as their highest level of education, 
and 37% indicated that their highest level was a bachelors degree. Pertain¬ 
ing to teaching experience, 77% of the teachers reported that they had more 
than 5 years teaching experience overall, and 71% reported that they had 
been teaching mathematics for more than 5 years. More than half (53%) of 
the participants indicated that they had received professional development 
in mathematics during the year prior to participating in the study. Addition¬ 
ally, 73% of the teachers worked at schools that offer Title I services. The 
largest percentage of teachers (38%) taught in rural schools, followed by city 
schools (32%), town schools (10%), and suburban schools (9%).' The largest 
group of participating teachers came from the South (43%), followed by the 
Northeast (39%), the Midwest (15%), and the West (3%). 2 There were no 
significant differences in teacher demographics between the experimental 
and control groups. 

Student participants. The initial student sample comprised 1,899 fifth 
grade students taught by the 79 teachers in the final sample. However, 
we omitted 461 students who had not responded to at least 50% of each 
of the student pre and post measures from the sample. The final student 
sample comprised 1,438 students. The experimental-group teachers 
taught 648 of the students, and the control-group teachers taught 790 of 
the students. 

The sample of students was equally split by gender. Regarding ethnic¬ 
ity, the majority of students identified themselves as White (65%), 12% 

1 Locales defined by the National Center for Education Statistics Common Core of Data (http://nces.ed.gov/ccd) 

2 Regions defined by the United States Census (http://www.census.gov/geo/www/us_regdiv.pdf) 
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identified as Black or African-American, and 13% identified as Asian, 
American Indian, Native Hawaiian, or other. Five percent of the student 
sample identified as Latino, and English was not the first language for 6% 
of the students. Eighty-seven percent of the students reported having a 
computer in their home. Additionally, 74% of the students reported using 
the computer generally a couple of times a week, 16% a couple of times a 
month and 7% a couple of times a year. With regard to computer use at 
school, 65% of the students reported using the computer a couple of times 
a week, 25% a couple of times a month, and 6% a couple of times a year. 
There were no significant differences in student demographics between the 
experimental and control groups. 

Instrumentation 

Due to the unavailability of measures with demonstrated validity and 
reliability that were sensitive to and aligned with the online professional 
development courses in fractions, algebraic thinking, and measurement, we 
developed research instruments to measure teachers’ pedagogical content 
knowledge and pedagogical practices as well as students’ achievement, and 
to record demographical data for both students and teachers. It should be 
noted that measures such as the Mathematical Knowledge for Teaching 
(MKT) instrument (Learning Mathematics for Teaching, 2008), though in 
development, were not available at the start of this study. 

Measure of Teachers’ Pedagogical Content Knowledge 

Teachers’ pedagogical content knowledge referred specifically to subject 
matter related to fractions, algebraic thinking, and measurement that 
formed the curriculum for the series of research courses. We assessed teach¬ 
ers’ pedagogical content knowledge using an instrument that was developed 
to measure the specific content topics covered across the three online profes¬ 
sional development courses. We used both multiple-choice and open-re¬ 
sponse items. We used a scoring rubric to score the open-response items and 
scored missing items as zero. 

The teachers’ pedagogical content knowledge measure comprised 31 
items. Nine items that assessed knowledge of fractions. These items focused 
on teachers’ knowledge of what fractions are and how they can be operated 
upon through engagement in mathematics as well as knowledge of specific 
mathematical models, particularly linear and area models. 

Sample items on the fractions subscale included: (a) “Describe both the 
importance and limitations of fraction algorithms, and when to use/teach 
them,” and (b) “Choose a model for teaching fractions. List one advantage 
and one disadvantage of teaching fractions with this model.” 

The algebraic thinking subscale included eight items designed to assess 
teachers’ knowledge of problems that can be used with students to develop 
their algebraic thinking as well as strategies for solving algebraic problems. 
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Table 1 . Correlation Matrix of Teachers’ Pedagogical Content Knowledge Measures 


Variables 

Pre Fractions 
Knowledge 

Post Fractions 
Knowledge 

Pre Algebraic 
Knowledge 

Post Algebraic 
Knowledge 

Pre Measure 
Knowledge 

Post Fractions Knowledge 

.612* 





Pre Algebraic Knowledge 

.204 

.309** 




Post Algebraic Knowledge 

.237*** 

.467* 

.365* 



Pre Measure Knowledge 

.316** 

.434* 

.489* 

.295** 


Post Measure Knowledge 

.281*** 

.506* 

.196 

.503* 

2/8*** 


Note: N = 79. * p < .001, ** p < .01, *** p < .05 


Sample items on the algebraic thinking subscale included: (a) “Give one 
example of a problem that encourages algebraic thinking,” and (b) “Give 
one example of a problem that highlights generalization as a way to solve 
algebraic problems.” 

The measure included 14 items assessing teachers’ knowledge of measure¬ 
ment. These items focused on teachers’ knowledge of tasks that encourage 
students’ understanding of what measurement is and why it is necessary, in 
addition to knowledge of how elementary students think about linear mea¬ 
surement, area, and capacity. Sample measurement items included: (a) “How 
would you explain differences between area and capacity to your students?” 
and (b) “Explain how the use of nonstandard objects can facilitate students’ 
understanding of measurement.” 

We calculated an overall knowledge score as well as three subscale 
scores for each teacher participant. We calculated scores using a weight¬ 
ed scale whereby all pedagogical content knowledge items were equally 
weighted with a maximum possible score of 1. We then calculated mean 
scores by summing the scores of all items within a scale and dividing 
by the total number of items for that scale. Higher scores were indica¬ 
tive of higher pedagogical content knowledge. The internal consistency 
reliability coefficient, as estimated by Cronbach’s alpha, for scores on 
the overall knowledge scale was .72 on the presurvey and .80 on the 
postsurvey. Cronbach’s alpha for scores on the subscale measures was 
significantly lower on the pre- and postsurveys respectively: fractions 
.54 and .48; algebraic thinking .55 and .60; and measurement .55 and 
.67. We performed Pearson’s product-moment correlation analyses to 
determine the relationships among the pedagogical content knowledge 
subscales. The correlation analyses confirmed statistically significant and 
positive relationships among the postsurvey pedagogical content knowl¬ 
edge subscales that ranged from .47 to .51. The presurvey subscales were 
also positive and statistically significant, with the exception of algebraic 
thinking and fractions, r(79) = .20, p = .072. Due to the lower reliability 
estimates of the individual subscales and the significant intercorrela¬ 
tions, we conducted analyses with the total scale scores. Table 1 presents 
the correlation matrix for measures of teachers’ pedagogical content 
knowledge. 
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Measure of Teachers’ Pedagogical Practices 

Teachers’ pedagogical practices in mathematics pertained to specific teach¬ 
ing practices in Fractions, Algebraic Thinking and Measurement that were 
promoted as best practices in the online professional development courses. A 
researcher-developed instrument was utilized to assess teachers’ pedagogical 
practices, and the course developers reviewed the instrument to confirm the 
accuracy of the content. All of the pedagogical practice items were measured 
on 4-point Likert scales that were anchored at: 1 = at least once a week and 4 
= never; 1 = strongly agree and 4 = strongly disagree; or 1 = always and 4 = 
rarely. The scores were summed and an average taken to yield three separate 
subscale scores (one for each subject area). Higher scores were indicative of 
better pedagogical practices. Less than 1% (0.73%) of data for the teachers’ 
pedagogical practices items were missing. A stochastic linear regression model 
was employed to impute this missing data for teacher practices. 

The pedagogical practices measure was composed of 38 items that as¬ 
sessed teachers’ self-reported pedagogical practices. Four items focused on 
Fractions, and sample items included “When teaching fractions to students, 
how often do you use area models?” and “When teaching fractions to 
students, how often do you use real world examples or situations of frac¬ 
tions?” Eight items measured pedagogical practices in Algebraic Thinking, 
and sample items included “How often do you apply ideas from the NCTM 
Algebra strand in your classroom practice?” and “How often do you use 
student interviewing techniques to explore student thinking?” Twenty-six 
items assessed teachers’ pedagogical practices in Measurement, and sample 
items included “When teaching measurement, how often do you teach your 
students how to measure the area of non-standard shapes?” and “When 
teaching measurement, how often do you teach your students how to use 
different tools and methods to measure capacity?” 

Cronbach’s alpha for scores on the overall pedagogical practices scale was 
.87 on the pre-survey and .90 on the post-survey. The reliability coefficient for 
scores on the Measurement Practices subscale was .82 for the presurvey and 
.88 for the postsurvey. Additionally, the Algebraic Thinking Practices subscale 
was .82 for both the pre- and postsurveys, but Cronbach’s alpha was lower for 
scores on the Fractions Practices subscale: .61on the presurvey and .60 on the 
postsurvey. We also conducted Pearson’s product-moment correlation analyses 
to investigate the intercorrelations among the pedagogical practices subscales. 
Pearson’s r was statistically significant and positive among the postsurvey 
pedagogical practices subscales and ranged from .28 to .55. The correlation 
analyses also confirmed statistically significant and positive relationships 
among the presurvey subscales, with the exception of Algebraic Thinking and 
Fractions, r( 79) = .16, p = .150. Due to the significant relationships and lower 
estimates of Cronbach’s alpha among the individual subscales, analyses were 
conducted with the total scale scores. Table 2 presents the correlation matrix 
for measures of teachers’ pedagogical practices. 
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Table 2. Correlation Matrix of Teachers’ Pedagogical Practices Measures 


Variables 

Pre Fractions 

Practices 

Post Fractions 

Practices 

Pre Algebraic 
Practices 

Post Algebraic 
Practices 

Pre Measure 

Practices 

Post Fractions Practices 

.438* 





Pre Algebraic Practices 

.163 

.150 




Post Algebraic Practices 

.013 

.276*** 

.328** 



Pre Measure Practices 

.461* 

.158 

.542* 

.245*** 


Post Measure Practices 

.288** 

.421* 

.304** 

.550* 

.564* 


Note: N = 79. * p < .001, ** p < .01, *** p < .05 


Measure of Students’ Mathematics Achievement 

To measure students’ achievement in fractions, algebraic thinking, and 
measurement, we developed tests that focused on these three subcontent 
areas specifically for this study. We constructed parallel pre and post mea¬ 
sures with the use of released test items from national and state standardized 
tests, as well as with test items that we created and piloted for this study. The 
selected test items targeted the specific student outcomes that were outlined 
in the research courses. We anticipated that teachers who completed the 
Fractions course would be better able to prepare students to: (a) understand 
the process of computations rather than simply use algorithms, (b) use mod¬ 
els to solve problems related to fractions, (c) find fractions of nonstandard 
shapes, and (d) solve traditional fractions problems. 

After completing the Algebraic Thinking course, teachers were expected 
to be better prepared to teach students to: (a) use representations such as 
graphs to draw conclusions, (b) solve missing value problems, (c) use gener¬ 
alization strategies such as finding a mathematical rule, (d) solve “unknown 
quantity” problems without solving for a variable, and (e) identify the equal 
sign as a balance point in an equation. 

The Measurement course was designed to prepare teachers to help 
students to: (a) understand the concepts of length and width; (b) differenti¬ 
ate area from capacity; (c) and use both standard and nonstandard units to 
determine length, area, and capacity. 

In an effort to minimize familiarity effects, we designed the pre- and 
posttests as parallel forms that comprised isomorphic items. Specifically, 
both instruments included the same balance of content areas and specific 
outcomes (skills) within those content areas. The parallel pre- and posttests 
each comprised 29 items. Specifically, each test included 8 fractions items, 

10 algebraic thinking items, and 11 measurement items. Each item had a 
maximum possible score of 1. The scores summed across items and aver¬ 
ages calculated to determine an overall knowledge score as well as a subscale 
score for each of the three content areas. Higher scores were indicative of 
higher pedagogical content knowledge. 

Regarding the overall mathematics achievement scale, Cronbach’s alpha 
was .73 on the pretest and .85 on the posttest. The reliability estimates, 

Volume 45 Number 1 | Journal of Research on Technology in Education | 15 
Copyright © 2012, ISTE (International Society for Technology in Education), 800.336.5191 
(U.S. & Canada) or 541.302.3777 (Int'l), iste@iste.org, iste.org. All rights reserved. 




Dash, Magidin de Kramer, O'Dwyer, Masters, & Russell 


Table 3. Correlation Matrix of Students’ Mathematics Knowledge Measures 


Variables 

Pre Fractions 

Practices 

Post Fractions 

Practices 

Pre Algebraic 
Practices 

Post Algebraic 
Practices 

Pre Measure 

Practices 

Post Fractions Practices 

.359* 





Pre Algebraic Practices 

.415* 

.434* 




Post Algebraic Practices 

.403* 

.575* 

.538* 



Pre Measure Practices 

.403* 

.427* 

.502* 

.532* 


Post Measure Practices 

.386* 

.538* 

.465* 

.635* 

.553* 


Note. N = 1438. * p < .001 


however, were lower for the subscales. Specifically, Cronbachs alpha for 
scores on the fractions subscale was .49 on the pretest and .68 on the post¬ 
test. The algebraic thinking subscale yielded a Cronbachs alpha of .60 on the 
pretest and .73 on the posttest. Cronbachs alpha for scores on the mea¬ 
surement subscale was .48 on the pretest and .59 on the posttest. Pearson’s 
product-moment correlation analyses confirmed statistically significant rela¬ 
tionships among the pretest subscales that ranged from .40 to .50. Regarding 
the posttest subscales, the correlations were also statistically significant and 
positive and ranged from .54 to .64. Due to the positive relationships and 
lower reliability estimates of the individual subscales, we conducted analyses 
with the total scale scores. Table 3 presents the correlation matrix for mea¬ 
sures of students’ mathematics achievement. 

Research Procedures 

We administered all of the research instruments online. Specifically, we 
designed a webpage for the research study that required teachers to sign in 
to a teacher management page to complete the teacher instruments. Teach¬ 
ers were allowed to complete their surveys in multiple sittings. However, 
once a teacher clicked the Submit button on any survey page, he or she was 
not allowed to return to that page to revise those answers. For example, if a 
teacher clicked Submit on page 3, he or she would have been taken to page 4. 
If the teacher exited the survey at that time, no answers from page 4 would 
have been submitted. The next time that teacher signed in, he or she would 
have been directed to page 4 and would not be able to revise any of the an¬ 
swers provided on pages 1 through 3. 

Teachers also used the teacher management page to administer the student 
instruments online. The student instruments were designed to be completed 
in a single class period, but students were allowed to finish a test in more than 
one sitting if necessary. Additionally, students were permitted to use calcula¬ 
tors or pencil and paper during the tests. Upon completion of the students’ 
tests, teachers could immediately access each student’s results. The rationale 
was that teachers could use the students’ test results as a formative assessment. 

In addition to instant feedback regarding students’ performance on the 
mathematics tests, we also offered teachers other incentives. Specifically, 
both experimental- and control-group teachers who completed all of the 
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study requirements were paid a stipend of $300. Teachers in the experimen¬ 
tal group were also able to receive graduate credit or continuing education 
units for the courses that they completed, and teachers in the control group 
were given the opportunity to complete a series of three online professional 
development courses after the research study ended. 

Results 

Online Professional Development and Teachers’ Pedagogical Content Knowledge 

We employed a repeated measures analysis of variance (RM-ANOVA) tech¬ 
nique to ascertain whether there were significant differences in participants’ 
pedagogical content knowledge (pre/post) as a function of group member¬ 
ship (experimental or control). The results of the RM-ANOVA confirmed a 
significant interaction between group membership (experimental or control) 
and total pedagogical content knowledge, F(l, 77) = 22.36 ,p < .001. This 
significant interaction suggests there was variability by group membership 
regarding pre to post gains in teachers’ pedagogical content knowledge. Spe¬ 
cifically, whereas there was an increase in knowledge scores for the experi¬ 
mental-group teachers from pre (M = .46, SD = .14) to post (M = .58, SD = 
.14), there was a slight decline for the control group from pre (M = .45, SD = 
.11) to post (M = .44, SD = .12). The effect size associated with this interac¬ 
tion as measured by rjp 2 was .23. Using Keppel and Wickens (2004) guide¬ 
lines, which suggest that .01-.05 is indicative of a small effect size, .06-. 14 
a medium effect size, and greater than .14 a large effect size, this coefficient 
was indicative of a large effect. 

Group membership (experimental or control) emerged as a significant 
main effect, F(l, 77) = 8.73, p = .004, confirming that teachers in the ex¬ 
perimental group received significantly higher overall pedagogical content 
knowledge scores (M = .52, SE = .02) than did teachers in the control 
group (M = .44, SE = .02). The effect size associated with this difference as 
measured by qp2 was .10 and is indicative of a medium effect. Total peda¬ 
gogical content knowledge (pre/post) also emerged as a significant main 
effect, F( 1, 77) = 16.31, p < .001, indicating that across all teachers, the 
overall mean score in total knowledge increased between administration of 
the pretest (M = .45, SD = .13) and the posttest (M = .50, SD = .15). The ef¬ 
fect size associated with this difference as measured by pp 2 was .18, which 
is large. However, it should be noted that this main effect shows differences 
pre to post for all teachers and therefore masks differences between the 
experimental and control groups. Table 4 (p. 18) presents the means and 
standard deviations for scores on the teacher pedagogical content knowl¬ 
edge scales by group membership (experimental or control). Table 5 (p. 

18) presents the summary statistics for the effects of online professional 
development on teachers’ pedagogical content knowledge by group mem¬ 
bership (experimental or control). 
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Table 4. Means and Standard Deviations for Scores on the Teachers’ Pedagogical Content Knowledge and Pedagogical Practices 
Scales by Group Membership 



Experimental Group (n = 34) 


Control Group (n = 45) 


Pre-Intervention Postintervention 

Pre-Intervention Postintervention 


M 

SD M 

SD 

M 

SD M SD 

Overall Pedagogical Content Knowledge 

.46 

.14 .58 

.14 

.45 

.11 .44 .12 

Overall Pedagogical Practices 

2.52 

.35 3.05 

.37 

2.71 

.37 2.79 .34 

Table 5. Summary Statistics for the Effects of Online Professional Development on Teachers’ Pedagogical Content Knowledge 
by Group Membership (Control or Experimental) 

Source 


df 

Sum of 
Squares 

Mean 

Square 

F tpf 

Within 






Time (Pre/Post) 


1 

.115 

.115 

16.314* .175 

Time x Group Membership 


1 

.158 

.158 

22.362* .225 

Error (Time) 


77 

.544 

.007 


Between 






Group 


i 

.229 

.229 

8.729** .102 

Error 


77 

2.018 

.026 



‘p <.001, ** p< .01 


Online Professional Development and Teachers’ Pedagogical Practices 

We also conduced repeated measures analysis of variance (RM-ANOVA) 
to determine whether there were significant differences by group mem¬ 
bership (control or experimental) with respect to participants’ pedagogi¬ 
cal practices (pre/post). The results of the analyses revealed a significant 
interaction between group membership (control or experimental) and 
pedagogical practices, F(l, 77) = 46.10, p < .001. The significant interaction 
is indicative of variability by group membership. Notably, participants in the 
experimental group showed significantly greater gains in scores for overall 
pedagogical practices from pre (M = 2.52, SD = .35) to post (M = 3.05, SD = 
.37), compared with control group teachers’ pre (M = 2.71, SD = .37) to post 
scores (M = 2.79, SD = .34). The effect size associated with this interaction as 
measured by rjp 2 was .38, which is quite large. 

Group membership (control or experimental) did not emerge as a sig¬ 
nificant main effect, F( 1, 77) = .230, p = .633, indicating that experimental- 
group teachers’ overall mean scores in pedagogical practices (M = 2.78, SE 
= .06) were not statistically significantly higher than overall mean scores for 
teachers in the control group (M = 2.75, SE = .05). However, the results in¬ 
dicated a significant main effect for overall pedagogical practices (pre/post), 
T(l, 77) = 84.53, p <.001, indicating that mean scores for teachers across 
both experimental and control groups increased from pre (M = 2.62, SD = 
.37) to post (M = 2.90, SD = .38). The effect size associated with this differ¬ 
ence as measured by qp2 was .52 and is indicative of a large effect. It should 
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Table 6. Summary Statistics for the Effects of Online Professional Development on Teachers’ Pedagogical Practices by Group 
Membership (Control or Experimental) 


Source 

df 

Sum of 
Squares 

Mean 

Square 

F 

>lf? 

Within 






Time (pre/post) 

1 

3.607 

3.607 

84.529* 

.523 

Time x Group Membership 

1 

1.967 

1.967 

46.103* 

.375 

Error (Time) 

77 

3.286 

.043 



Between 






Group Membership 

1 

.049 

.049 

.230 


Error 

77 

16.483 

.214 




* p <.001 


Table 7. Means and Standard Deviations for Scores on the Students' Mathematics Knowledge Scales 


Experimental Group (n = 648) Control Group (n = 790) 


Pre-Intervention 

Postintervention 

Pre-Intervention 

Postintervention 

M 

SD 

M 

SD 

M 

SD 

M SD 

.42 

.16 

.54 

.21 

.44 

.16 

.53 .20 


be noted that this significant main effect for overall pedagogical practices 
(pre/post) masks differences between the experimental and control groups. 
Table 6 presents the summary statistics for the effects of online professional 
development on teachers’ pedagogical practices by group membership (ex¬ 
perimental or control). 

Online Professional Development and Students’ Mathematics Achievement 

Table 7 shows the means and standard deviations for students’ scores 
on the mathematics knowledge subscales and overall scale. We applied 
hierarchical linear models to determine whether teachers’ assignment to 
the experimental or control group was a significant predictor of students’ 
posttest scores after controlling for students’ pretest scores. We used this 
type of analysis to account for the nesting of students within randomly 
assigned teachers and allowed for the estimation of the impact of teach¬ 
ers’ assignment to the experimental or control conditions on students’ 
mathematics achievement. Students’ posttest scores were modeled as a 
function of their pretest scores (Level 1) and as a function of teachers’ 
group membership (Level 2). 

The results of the unconditional model indicated significant variation 
in students’ posttest knowledge scores among teachers. Specifically, the un¬ 
conditional intraclass correlation coefficient (ICC) was .32 for the overall 
mathematics posttest scores. This indicates that 26-32% of the total variabil¬ 
ity on the posttest scores could be attributed to between-teacher differences. 
These nonzero ICCs, in addition to the modest sample size, support the 
need for the use of hierarchical linear regression models to estimate unbi¬ 
ased treatment effects. 
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Table 8. HLM Results for the Effects of Teachers’ Group Membership (Control or Experimental) on Students’ Mathematics Achievement 



Students’ Overall Mathematics Scores 

Unconditional Model 



Fixed Components 

Coefficient 

.537 

s.e. Sig. 

.014 < .001 

Random Components 

Within School 

Between School ICC 

.029 

.014 .32 

Conditional Model 

Coefficient 

s.e. Sig. 

Intercept 

.523 

.009 < .001 

Student Measures 



Pretest Scores 

.806 

.028 < .001 

Teacher Measures 



Group membership (0 = Control, 1 = Treatment) 

.027 

.014 .062 

Random Components 


Residual Variance 

Within Schools 

.019 


Between Schools 

.003 

p<.001 

Total Residual 

.022 


Total Variance Explained by the Model 

48.13% 



The results of the HLM analyses reported in Table 8 indicated that after 
controlling for students’ pretest scores, teachers’ participation in the online 
professional development treatment did not predict students’ achievement 
in mathematics (b = .027, t = 1.89, df= 77, p = .062). Essentially, there were 
no significant differences in students’ mathematics achievement, as a func¬ 
tion of teachers’ group membership (control or experimental). 

Discussion 

Key Findings 

This research study examined the impact of online professional develop¬ 
ment courses in fractions, algebraic thinking, and measurement on fifth 
grade teachers’ pedagogical content knowledge and pedagogical practices 
as well as their students’ mathematics achievement. The results of the study 
showed that teachers assigned to the experimental group had significant 
gains in scores of overall pedagogical content knowledge, whereas these 
knowledge scores for the control group actually declined slightly. Addi¬ 
tionally, the experimental group teachers had significantly greater gains in 
scores for pedagogical practices related to overall mathematics, compared 
with teachers in the control group. Moreover, the effect sizes associated 
with these differences were large for gains in both knowledge and practic¬ 
es. These findings provide evidence and support for the research literature, 
which indicates that professional development for teachers can be effective 
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if it is intensive, sustained, and content focused (Yoon et al, 2007). Teach¬ 
ers in this study participated in a minimum of approximately 24 hours of 
professional development for each course, which equates to more than ap¬ 
proximately 70 hours of professional development across the three online 
professional development courses. Furthermore, the online professional 
development courses were based on a learning community model that 
allowed teachers to actively explore students’ conceptions and misconcep¬ 
tions about mathematics through various activities, such as student inter¬ 
views, analyses of students’ work, and peer-to-peer discussions. Moreover, 
consistent with the tenets of Shulman (1986), teachers were provided with 
strategies for representing and formulating mathematical concepts in a 
way to make the subject matter more comprehensible to upper elementary 
students. Overall, the results of the study indicate that intensive, sustained, 
content-focused online professional development in mathematics can 
effect positive change in teachers’ pedagogical content knowledge and 
pedagogical practices. 

Limitations of the Study 

One significant limitation of this study is the high attrition rate. Despite pro¬ 
viding teachers with a moderate stipend and free access to online professional 
development (teachers in the control group were allowed to take each course 
free of charge upon completion of the study), a large number of participants 
did not persist through the full study. However, it should be noted that, unlike 
typical professional development programs, teachers in this study were required 
to commit to three semesters of participation. In a “real world” context, teachers 
complete individual courses and are not usually required to complete a series 
of three professional development courses. Therefore, the high attrition rate 
associated with this research study should not generalize to a standard online 
professional development program. Nevertheless, although the number of 
teachers who completed the study was sufficient for data analyses, it is possible 
that effects might have differed for teachers who dropped out versus those who 
persisted. Such differences may bias the estimates reported herein. Chi-square 
analyses revealed that the only significant difference in teacher demographics 
between teachers who dropped out and those who remained in the study was 
previous experience with online professional development. Specifically, teach¬ 
ers who dropped out of the study had less experience with online professional 
development. Yukselturk and Inan (2006) suggest that participants sometimes 
underestimate the time and effort required for successful completion of online 
courses and can experience anxiety regarding their technology proficiency. It 
should be noted that for this study, the first session (the first week of each of the 
online professional development courses) was an orientation session. However, 
the results suggest that the orientation session of online professional develop¬ 
ment programs ought to be as carefully planned as the content sessions to allay 
participants’ anxieties and help set realistic expectations. 
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Regarding students’ mathematics achievement, the results of the HLM 
analyses indicated that positive changes in teachers’ pedagogical content 
knowledge and practices did not translate to any meaningful differences for 
students. Specifically, there were no statistically significant differences in 
students’ overall mathematics achievement as a function of teachers’ group 
membership (control or experimental). It would be easy to conclude that 
teacher professional development has no impact on student achievement, 
especially given the findings of other large-scale studies, such as those con¬ 
ducted by the American Institutes for Research on the impact of professional 
development in middle school mathematics and early reading instruction 
(Garet, Cronen, Eaton, Kurki, Ludwig, Jones, Uekawa, Falk, Bloom, Doo¬ 
little, Zhu, & Sztejnberg, 2008; Garet, Wayne, Stancavage, Taylor, Walters, 
Song, Brown, Hurlburt, Zhu, Sepanik, & Doolittle, 2010). However, it 
should be noted that the true effects of professional development on student 
achievement cannot be ascertained without first considering teachers’ op¬ 
portunity to implement course material. 

It should be noted that for this study, we administered the student 
measures in a relatively short period after the professional development 
ended. Therefore, teachers would have had a limited opportunity to imple¬ 
ment the content from the online professional development courses in their 
classrooms. Additionally, the sample of teachers was drawn from multiple 
states, and as there is no national curriculum, the schedule of the online 
professional development courses could not be aligned with the mathemat¬ 
ics syllabi of individual teachers. Therefore, it is possible, for example, that 
a teacher might have participated in the Fractions course after that subject 
matter had already been taught to students. Consequently, the teacher would 
not be able to implement the new knowledge, skills, and strategies he or 
she would have learned from that Fractions with that group of students. In 
specifying the link between professional development and student achieve¬ 
ment, Yoon et al. (2007) indicated that if a teacher fails to apply new ideas 
from professional development to classroom instruction or arguably has 
limited opportunity to fully implement those new ideas, improved student 
learning cannot be expected. 

Recommendations for Future Research 

The implications regarding the implementation of course material, coupled 
with the research literature that indicates change in teacher practices takes 
time, underscore the need for a follow-up study to determine whether 
teacher gains persist over time and affect student learning. Future research 
should consider the follow-up support system needed as well as teachers’ 
motivation to apply professional development to classroom teaching. Ad¬ 
ditionally, future research studies can be strengthened by adding a face-to- 
face component, thereby offering a hybrid intervention that includes both 
online and face-to-face professional development. Such a research design 
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might allow comparisons to be made between online professional develop¬ 
ment and face-to-face professional development. 

Indeed, a complete depiction of what teachers actually do in their class¬ 
rooms requires data collection methods such as classroom observations, 
which were not feasible for this study. However, Hamilton and Guarino 
(2004) argue that it is possible to gain accurate information regarding 
teachers’ instructional practices through self-reported questionnaires. Still, 
self-reported measures of teachers’ pedagogical practices, as were used in 
this study, have the limitation of providing information regarding teachers’ 
reports of their instructional practices rather than their actual practices. 

Implications for Practice 

Despite the shortcomings, the findings of this study add to the growing body 
of research regarding the impact of professional development in general and 
online professional development in particular. The results of this study hold 
promise that online professional development can be an effective strategy to 
improve teachers’ pedagogical content knowledge and pedagogical practices. 
However, the impact of online professional development on students’ aca¬ 
demic achievement, though null for this study, requires further investigation 
if a more contextualized picture regarding the efficacy of online professional 
development is to be depicted. 
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